The human-computer interface is an important aspect of information technology. The transfer of information from human user to information system (for example, by keyboard or mouse) and the transfer of information from information system to human user (for example, by visual display or printer) are critical for efficient human use of the information system. Engaging one or more of the human senses accomplishes information transfer to the user. Visual display is the most common output of current computer systems (computer monitors, printer, lights on an instrument panel, etc.), but other sensory stimuli are increasingly being used, for example audio output of synthesized or recorded speech, alarm bells, and the like. Having more than just visual sensory output available allows a designer to more closely match the system output to the needs of a particular situation. In some cases a combination of output formats (for example, simultaneous video and audio output) will be more effective than any single format.
Human-computer interface outputs, which may more generically be called display formats, have more recently come to include those which engage the human sense of touch, both the detailed tactile sense of touch (Braille displays) and the forces involved in large body movements (force feedback or haptic systems). Braille is used to provide a reading experience for blind and visually impaired users. Haptic systems are typically used in virtual reality applications that engage multiple senses for more realistic recreation of a virtual world, and for real-world application such as remote control of robots (teleoperation).
Braille output from information systems is available in two formats: non-refreshable and refreshable. An example of a non-refreshable output is a printed (embossed) sheet of paper that has patterns of raised dots pressed into the paper to represent Braille characters. Printed Braille uses a consumable material (the paper) that is not reused. Refreshable Braille has a surface with dots that can be raised or lowered under computer control to form the Braille characters, so that the same surface can be reused many times for different text, comparable to the way in which a visual computer monitor can be used to display many images on the same surface without consuming any materials.
Braille is suitable for the reading of text, and alternative Braille codes have been devised to allow reading of mathematical expressions and musical notation. However, many blind and visually impaired individuals also have a need for touch-based or tactile graphics, in which the sense of touch can be used to experience two-dimensional or three-dimensional imagery. Tactile graphics is most commonly used to portray line drawings, textures, and to some extent generic pictures, where differences in tactile characteristics are used to portray differences in an image. Tactile graphics is important in learning (for example, graphs of mathematical functions), in design (for example, artistic or engineering design work), and in entertainment and art appreciation. Many web pages convey important information in an image format, which can be represented by tactile graphics. Tactile graphics is currently available in non-refreshable format. A number of commercially available devices can produce tactile graphics by pressing a pattern into a heavy sheet of paper or plastic, by heating a sheet of plastic that deforms when heated to form a raised pattern, or by depositing a material onto a sheet to form a pattern. All of these methods are effective, but they consume materials for every page printed. Non-refreshable tactile material is very bulky compared to visual printed material, and storage of a large number of printouts becomes a problem. Furthermore, the cost of materials is substantial, making applications that require many images (such as design, study, and web browsing) very expensive. These problems could be avoided by the availability of a refreshable tactile graphic display that would allow the same tactile surface to be used over and over for many tactile graphic images. Unfortunately, using the conventional method for tactile output, this would require a two-dimensional array large enough to show reasonably complex images, and a high enough area density of pins or other stimulus devised to provide an acceptable level of realism. For example, the user should be able to feel a displayed straight or curved line, and be able to mentally interpret it as a continuous straight or curved line rather than a collection of unrelated points—an example might be ten to twenty pins per linear inch, with several thousand pins as a minimum count for the entire display. Again by conventional design, a separate actuator drives each pin (commercial Braille displays, for example, use piezoelectric actuators), and the cost of thousands of very small actuators (at least several dollars each) puts the possibility of tactile graphic display traditionally designed beyond the financial means of almost all potential users.
The detailed human sense of touch is a scanning process. If a fingertip or other body part rests motionless on a surface, only a low resolution tactile image is perceived by the brain. If a fingertip slides across a surface, however, the changes in pressure against the skin caused by irregularities of the surface being felt scan across the surface of the skin, and the brain uses these perceived tactile impressions, combined with knowledge of the motion of the finger or the surface, to build up a mental model of the extended surface.
Since, heretofore, the scanning nature of the human sense of touch has meant that an extended surface display is necessary for the representation of an extended object, given the attendant difficulties noted above for extended graphic displays of conventional design further development could be utilized and this work has begun. For example, fixed displacement-based technology by Orbital Research uses microvalves that direct air to inflate small elastic balloons intended for use in an extended display for display of Braille (in a fairly low-density array). The display would, of course, remain static as the user reads it.
Johns Hopkins University has constructed a refreshable fingertip tactile graphic display for research on the sense of touch. The display uses a 20 by 20 pin array of stimulus points at 0.5 mm spacing, with a large multi-position linear actuator for each pin, and mechanical concentrator (linkages) to direct the activity of all 400 actuators into a small area (approximately 1 cm square, the entire device being several feet on each side). The action of the display is based upon controlled displacements of the pins at selected heights. The motors used to drive the pins have optical feedback to determine position, but this is mechanically linked to the driving source. The design is bulky and expensive, with pin operation based on fixed displacement.
Devices intended for accessibility applications have also been heretofore suggested and/or utilized. The VIRTOUCH VIRTUAL MOUSE uses three fixed displacement-based (4-level selection) 8×4 actuator fingertip arrays mounted on a computer mouse device. The arrays are driven by piezoelectric actuators, and while a user can learn to interpret tactile information (such as the tracking of the cursor across a virtual line), the spacing of the pins is several times too large to permit it to create detailed tactile sensations that would be needed for a general-purpose tactile display. Likewise, the OPTACON by Telesensory produces low resolution images using an optical scanner and a low-density array of vibrating pins to permit blind and visually impaired users to read printed or displayed text scanned by the device. The user must learn to interpret the vibration of the pins as image information (i.e., the pins do not change elevation). The use of fairly large electromechanical actuators and mechanical linkages limits the resolution of images achievable by the device.
Thus there remains a need for affordable refreshable tactile graphics for use by blind and visually impaired users, and particularly in a relatively portable and less cumbersome deployment. If the technology could also be used to improve the sense of touch in virtual reality and other related mainstream applications, this would increase the market size for the technology and drive further commercial development that would improve quality and lower the cost for both mainstream and accessibility applications.